Published on Jun 05, 2023
For real time facial animation on the web, following are the important requirements:
1). Easy installation: like in most of the application, the virtual presenter is not the most important attraction of the proposed services. For this, we would not use any plug-in and the complete applet development (Shout3D rendering + deformation engine) is pure java.
2). Visual quality: in case of the virtual talking heads on the web looks and realism are of utmost importance. In the case of the virtual cartoon-like character, we can be freer to use artistic and exaggerated deformation. In either case the visual quality must be good.
3). Fast download: to reduce the delay of downloading model, we need to develop low resolution and less complex model (e.g. use of compressed model from Shout3D) and compress the size of data needed for animation (FAT). Some download sizes of files are described later.
4). Real time interactivity: our virtual presenter must be able to interact in different ways depending upon the user. In this case, video streaming or preprocessing is not possible.
Thus, parameterized animation of the virtual presenter is of importance, and we choose MPEG-4 FAP as these parameters. One example of this interactivity is under development for the IST Interface project, where a Dialogue Manager, a TTS and a Phoneme/Bookmark converter provide real time dialogue and animation. - Easy web integration: integration of this applet in web pages should be easy and must allow communication between the applet and the html pages. This communication, to select a model and some visual parameters, uses JavaScript. The current implementation of the MPEG-4 facial animation player through FAT is written completely in java and uses the Shout3D rendering engine.
This engine was developed in Java in order to simplify the web integration (no plug-in is necessary). But, Shout3D provides two types of rendering engines based on the same APIs. The first is pure java that can be use without a plug-in installation, but the performance is linked with the complexity of the models.
